Algebraic relaxations and hardness results in polynomial optimization and Lyapunov analysis

The contributions of the first half of this thesis are on the computational and algebraic aspects of convexity in polynomial optimization. We show that unless P=NP, there exists no polynomial time (or even pseudo-polynomial time) algorithm that can decide whether a multivariate polynomial of degree four (or higher even degree) is globally convex. This solves a problem that has been open since 1992 when N. Z. Shor asked for the complexity of deciding convexity for quartic polynomials. We also prove that deciding strict convexity, strong convexity, quasiconvexity, and pseudoconvexity of polynomials of even degree four or higher is strongly NP-hard. By contrast, we show that quasiconvexity and pseudoconvexity of odd degree polynomials can be decided in polynomial time. We then turn our attention to sos-convexity—an algebraic sum of squares (sos) based sufficient condition for polynomial convexity that can be efficiently checked with semidefinite programming. We show that three natural formulations for sos-convexity derived from relaxations on the definition of convexity, its first order characterization, and its second order characterization are equivalent. We present the first example of a convex polynomial that is not sos-convex. Our main result then is to prove that the cones of convex and sos-convex polynomials (resp. forms) in n variables and of degree d coincide if and only if n = 1 or d = 2 or (n, d) = (2, 4) (resp. n = 2 or d = 2 or (n, d) = (3, 4)). Although for disparate reasons, the remarkable outcome is that convex polynomials (resp. forms) are sosconvex exactly in cases where nonnegative polynomials (resp. forms) are sums of squares, as characterized by Hilbert in 1888. The contributions of the second half of this thesis are on the development